A Comparison of x86 Computer Architecture Simulators

Simulation is used as a primary performance evaluation methodology in most computer architecture publications. There is not much literature dealing with the evaluation of simulators by comparing them to each other and to the state-of-the-art processors. The absence of performance validation of simulators may cause experimental errors that can lead to incorrect conclusions. This work provides a simulation accuracy and performance comparison of four modern x86 computer architecture simulators: gem5 [1], Sniper [2], MARSSx86 [3] and ZSim [4]. We configured these simulators to model one of Intel’s high-performance processor, Core-i7 Haswell microarchitecture based CPU. Then we quantified the experimental errors. The selected simulators for this study have diverse design strategies with respect to detail and abstraction. All of them are contemporary simulators with active development. Validation effort for gem5 simulator exists for ARM based systems [1, 5], however; no validation for x86 systems exists. Latest validation effort for Sniper was done for Intel Nehalem microarchitecure processor showing a single-core error of 11.1% for a subset of SPLASH-2 benchmarks [2]. ZSim has been validated against an Intel Westmere system showing an average error of 10% [6]. We did not find any rigorous validation effort for MARSSx86. Our previous work compared the accuracy and speed of some of the aforementioned simulators, and few others, for single core simulations only [7]. This work, compares more simulators for singlecore and multicore runs with real hardware platform runs. The experimental system that we used to test the four simulators is similar to Haswell microarchitecture (i7-4770 CPU, 3.40 GHz), see Table 1. As all the exact configurations for our target processor are not published by Intel, we tried our best to model similar features based on some Intel documentation [8] and other sources [9, 10, 11]. We compared the instruction per cycle (IPC), L1 data cache miss, L3 cache miss and branch misprediction values from the simulation results with that of real hardware runs for MiBench and SPEC-CPU2006 benchmark suites. Figure 1 shows single-core’s IPC, branch mispredictions and L1 data cache misses, and dual-core’s and quad-core’s IPC runs normalized to results obtained on real hardware, using hardware monitoring counters. For dual and quadcore runs, benchmark combinations are randomly selected from SPEC-CPU2006 benchmark suite. The figure shows